Image processing includes image recognition processing for recognizing an object in an image. In this case, however, image capturing processing includes a degradation such as noise during capturing. Accordingly, the image recognition may require image restoration processing in some cases (e.g., see PTL 1). An example of the restoration processing is a learning-based super-resolution technique.
When the image recognition or the learning-based super-resolution is performed on an object in an image, an image processing device for processing an image calculates geometric transformation between a posture of an object in an input image and a posture of a captured object in a reference image. This processing is generally called “image registration (hereinafter also referred to simply as “registration”)”. Lucas-Kanade method (e.g., see NPL 1) is known as an image registration method. The Lucas-Kanade method is a method for registration of images. In the method, sum of squares of differences between pixel values of the object in the input image and pixel values in the reference image in which the image registration is performed, is minimized. In the Lucas-Kanade method, positions having the minimized sum are calculated by a gradient method. However, the Lucas-Kanade method is based on a premise that, for example, the input image and the reference image are substantially the same images like adjacent frames in a video footage. Accordingly, the Lucas-Kanade method cannot be applied when a resolution or a pattern (e.g., a variation of the number of license plates) of the input image is considerably different from that of the reference image.
Thus, there is an issue that the techniques described in PTL 1 and NPL 1 cannot be applied when there are many variations of patterns in the input image and the reference image, like images captured by a surveillance camera.
In this regard, an AAM (Active Appearance model) has been proposed (e.g., see PTL 2 and NPL 2) as a method capable of performing registration of images even when there are variations of image patterns. The AAM learns a distribution of variations of patterns from a plurality of reference images by using a principal component analysis. The AAM approximates the variation distribution with, for example, an anisotropic Gaussian distribution. Thus, the AAM is a method of simultaneously performing registration of the reference image and the input image and estimation of patterns.